DRAFT: do not set LIBRARY_PATH for LLVM and adapt compiler config to insert paths instead - #4089
DRAFT: do not set LIBRARY_PATH for LLVM and adapt compiler config to insert paths instead#4089Thyre wants to merge 4 commits into
LIBRARY_PATH for LLVM and adapt compiler config to insert paths instead#4089Conversation
Thyre
changed the title
LIBRARY_PATH for LLVM and adapt compiler config to insert paths instead
|
Test report by @Thyre Overview of tested easyconfigs (in order)
Build succeeded for 0 out of 1 (total: 30 mins 29 secs) (1 easyconfigs in total) |
Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>
c44fcef to
3e778c8
Compare
|
Test report by @Thyre Overview of tested easyconfigs (in order)
Build succeeded for 0 out of 1 (total: 2 hours 9 mins 24 secs) (1 easyconfigs in total) |
|
The multi-stage build failed because the compiler config file now picks up Stage 2 during the Stage 3 build, hence causing So we really need to only add the compiler paths for the final installation. |
Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>
|
Test report by @Thyre Overview of tested easyconfigs (in order)
Build succeeded for 0 out of 1 (total: 6 hours 29 mins 39 secs) (1 easyconfigs in total) |
Signed-off-by: Jan André Reuter <j.reuter@fz-juelich.de>
|
lld really did not like |
|
Test report by @Thyre Overview of tested easyconfigs (in order)
Build succeeded for 0 out of 1 (total: 6 hours 29 mins 0 secs) (1 easyconfigs in total) |
|
Still hitting the same failure: Wondering if I hit some Clang driver bug here. We certainly only specify |
| # Add flags to make sure LLVM picks up its own directories correctly. | ||
| # This ensures that LLVM finds it own stuff, even when LIBRARY_PATH is not set. | ||
| opts.append(f'-L{installdir}/lib') | ||
| opts.append(f'-Wl,-rpath={installdir}/lib') |
There was a problem hiding this comment.
I think if we are using rpath wrappers this is redundant
There was a problem hiding this comment.
The main point here is that people might still use LLVM without rpath wrappers (or a buildenv module), but we still want to make sure we're picking up the correct libraries.
| runtime_libdir = self.get_runtime_lib_path(installdir, fail_ok=True) | ||
| if os.path.exists(runtime_libdir): | ||
| opts.append(f'-L{installdir}/{runtime_libdir}') | ||
| opts.append(f'-Wl,-rpath,{installdir}/{runtime_libdir}') |
There was a problem hiding this comment.
Same as above, even if we are not setting LIBRARY_PATH we should still have LD_LIBRARY_PATH when not using rpathing so these are probably not needed
There was a problem hiding this comment.
What about EESSI, where LD_LIBRARY_PATH is filtered?
I also want to make sure that we do not accidentally pick up a wrong libomp.so, e.g. from ROCm-LLVM when LLVM is loaded.
|
I'm wondering if this could be more easily solved by appending the LIBRARY_PATH for CORE compilers instead of prepending it so that even if the load order is something like
we end up with the OFC this works if we have only 1 level of core compilers. |
|
I guess an LMOD alternative could also be something like
|
|
Didn't look into this in detail, but in some places we may assume that |
Agreed that we need to be careful here, and ensure that the correct rpath is still set. But, given that I haven't managed to get a single successful build due to the LLVM tests, I'm not sure if this is the right approach going forward... easybuilders/easybuild-easyconfigs#26267 will solve the easiest case to run into mixing LLVM versions, Mesa. For now, I consider this to be the better approach. |
|
Given the failure we saw with GCC 14.2.0 because of a missing Instead, we should try to explicitly link LLVM as a static library whenever possible. That gets rid of the hard dependency, removing the chance that libraries clash. With that, I'll go ahead and close the PR. If this comes up again, we can use this as a starting point, if necessary. |
3 participants
(created using
eb --new-pr)LLVM is a major part of our toolchains, especially since we started to use a single LLVM installation including all bells and whistles. However, this can lead to incorrect libraries being picked up when multiple LLVM-based applications are loaded.
In the case of ROCm-LLVM and LLVM, I encountered issues when e.g. Mesa & ROCm-LLVM are loaded. Mesa has a dependency on LLVM, hence polluting
LIBRARY_PATHwith its libraries. When we then try to build an application with ROCm-LLVM, we potentially pick up the incorrect OpenMP libraries (and other LLVM ones).For Mesa, this could be solved easily: Move LLVM to being included statically, hence moving LLVM to a build dependency. However, a long time solution would be desirable.
This PR tries to achieve this by removing
LIBRARY_PATHfrom LLVM altogether, as suggested by @ocaisa. We do a similar approach with GCC.This however needs very thorough testing. I'm very confident that LLVM knows where to find its stuff, but less confident about applications using just some parts of LLVM. They should rely on what
llvm-configor LLVM's CMake scripts provide, but you never know.Unfortunately, only adapting the compiler configuration is insufficient. LLVM rpaths
LIBRARY_PATHbefore any arguments passed by the user (including config files), so we cannot reliably influence which libraries are picked up at the end.